The government at a Cabinet meeting Tuesday adopted its first basic plan on the development and utilization of artificial intelligence. The basic plan stipulates that Japan will create reliable AI while balancing technological innovation and risk management, with an aim to become a country that offers the best environment for AI development and utilization. Japan lags behind not only other advanced nations but also countries with smaller economies in terms of AI development, and the gap is becoming wider year by year, it warns. In a time of both misinformation and too much information, quality journalism is more crucial than ever. By subscribing, you can help us get the story right.

Ting Cai, head of Rakuten Group's artificial intelligence team, has the task of creating AI systems that would augment the company's many businesses at a minimal cost. Rakuten Group is expanding its AI team under the stewardship of a Google veteran and building models with a focus on cost efficiency. Ting Cai, now three years into his tenure at the head of the e-commerce pioneer's artificial intelligence team, has the task of creating AI systems that would augment the company's many businesses and support the handling of commercial transactions at a minimal cost. He oversees a team that's grown to 1,000 this year and has a battery of "thousands" of Nvidia chips to work with. Tokyo-based Rakuten is wrestling with a struggling mobile business and constant competition in online shopping, both of which could get a significant boost from effective deployment of new AI tools.

Deep learning has shown impressive results obtained at the cost of training huge neural networks. However, the larger the architecture, the higher the computational, financial, and environmental costs during training and inference. We aim at reducing both training and inference durations. We focus on Neural Architecture Growth, which can increase the size of a small model when needed, directly during training using information from the backpropagation. We expand existing work and freely grow neural networks in the form of any Directed Acyclic Graph by reducing expressivity bottlenecks in the architecture. We explore strategies to reduce excessive computations and steer network growth toward more parameter-efficient architectures.

In continual learning (CL), model growth enhances adaptability over new data, improving knowledge retention for more tasks. However, improper model growth can lead to severe degradation of previously learned knowledge, an issue we name as growth-induced forgetting (GIFt), especially in task-agnostic CL using entire grown model for inference. Existing works, despite adopting model growth and random initialization for better adaptability, often fail to recognize the presence of GIFt caused by improper model growth. This oversight limits comprehensive control of forgetting and hinders full utilization of model growth. We are the first in CL to identify this issue and conduct an in-depth study on root cause of GIFt, where layer expansion stands out among model growth strategies, widening layers without affecting model functionality. Yet, direct adoption of layer expansion presents challenges. It lacks data-driven control and initialization of expanded parameters to balance adaptability and knowledge retention. This paper presents a novel SparseGrow approach to overcome the issue of GIFt while enhancing adaptability over new data. SparseGrow employs data-driven sparse layer expansion to control efficient parameter usage during growth, reducing GIFt from excessive growth and functionality changes. It also combines sparse growth with on-data initialization at training late-stage to create partially 0-valued expansions that fit learned distribution, enhancing retention and adaptability. To further minimize forgetting, freezing is applied by calculating the sparse mask, allowing data-driven preservation of important parameters. Through experiments across datasets with various settings, cases and task numbers, we demonstrate the necessity of layer expansion and showcase the effectiveness of SparseGrow in overcoming GIFt, highlighting its adaptability and knowledge retention for incremental tasks.

Continual learning (CL) refers to the ability of an intelligent system to sequentially acquire and retain knowledge from a stream of data with as little computational overhead as possible. To this end; regularization, replay, architecture, and parameter isolation approaches were introduced to the literature. Parameter isolation using a sparse network which enables to allocate distinct parts of the neural network to different tasks and also allows to share of parameters between tasks if they are similar. Dynamic Sparse Training (DST) is a prominent way to find these sparse networks and isolate them for each task. This paper is the first empirical study investigating the effect of different DST components under the CL paradigm to fill a critical research gap and shed light on the optimal configuration of DST for CL if it exists. Therefore, we perform a comprehensive study in which we investigate various DST components to find the best topology per task on well-known CIFAR100 and miniImageNet benchmarks in a task-incremental CL setup since our primary focus is to evaluate the performance of various DST criteria, rather than the process of mask selection. We found that, at a low sparsity level, Erdos-R\'enyi Kernel (ERK) initialization utilizes the backbone more efficiently and allows to effectively learn increments of tasks. At a high sparsity level, unless it is extreme, uniform initialization demonstrates a more reliable and robust performance. In terms of growth strategy; performance is dependent on the defined initialization strategy and the extent of sparsity. Finally, adaptivity within DST components is a promising way for better continual learners.

Large language models (LLMs) have achieved remarkable success in NLP and multimodal tasks, among others. Despite these successes, two main challenges remain in developing LLMs: (i) high computational cost, and (ii) fair and objective evaluations. In this paper, we report a solution to significantly reduce LLM training cost through a growth strategy. We demonstrate that a 101B-parameter LLM with 0.31T tokens can be trained with a budget of 100K US dollars. Inspired by IQ tests, we also consolidate an additional range of evaluations on top of existing evaluations that focus on knowledge-oriented abilities. These IQ evaluations include symbolic mapping, rule understanding, pattern mining, and anti-interference. Such evaluations minimize the potential impact of memorization. Experimental results show that our model, named FLM-101B, trained with a budget of 100K US dollars, achieves performance comparable to powerful and well-known models, e.g., GPT-3 and GLM-130B, especially on the additional range of IQ evaluations. The checkpoint of FLM-101B is released at https://huggingface.co/CofeAI/FLM-101B.

A New Market study by Report Consultant on the Global Artificial Intelligence (AI) Software Market Report offers an in-house analysis of global economic conditions and related economic factors and indicators to evaluate their impact on the Artificial Intelligence (AI) Software market historically to propose a tentative future scenario and current growth traits. This detailed report on Keyword market largely focuses on prominent facets such as product portfolio, payment channels, service offerings, applications, in addition to technological sophistication. The report covers key developments in the Artificial Intelligence (AI) Software Market as organic and inorganic growth strategies. Various companies are focusing on organic growth strategies such as launches, product approvals and others such as patents and events. Inorganic growth strategies activities witnessed in the market were acquisitions, partnership & collaborations.

Hillenbrand, Inc.'s acquisition of LINXIS Group, leaders in specialized equipment for the food, pharma, and cosmetics industries, has formally been completed. With a global footprint, LINXIS Group specializes in the design, installation and service of industrial process equipment and automation solutions that are complementary to the equipment and solutions Coperion currently offers to the food and pharma industries. LINXIS Group will be a part of the Coperion Food, Health & Nutrition Division, headed by Kevin Buchler as President Food, Health and Nutrition division and Tim Cook as Chief Strategy and Business Development Officer, Linxis. Both companies will combine their strengths and capabilities as global suppliers to the food and health industries. LINXIS Group's highly specialized equipment and strong reputation as a leading supplier for mixing, ingredient automation, and portioning solutions directly align with Coperion's growth strategy for food applications.

As the world's most valuable company, with a $2T market cap and over $100B in annual profit, Apple has long been known for its cell phones, laptops, tablets, and watches. Today, the company is investing aggressively in high-tech areas like AR/VR, AI, and semiconductors to lay the groundwork for products and features that affect health & wellness, mobility, digital connections, and more. In recent years, companies in the AR/VR space have become serious M&A targets for Apple, coinciding with rumors about its internal development of a mixed reality headset that is expected to release in 2023 (eventually followed by a sleeker pair of glasses). While Apple has not been explicit about its metaverse ambitions like fellow big tech giant Meta, its continued investment in AR/VR sets it up for more immersive digital environments of the future -- and the hardware they may need. II-VI makes the optical technologies that power features like Face ID and Portrait mode, as well as the lasers used in Apple's LiDAR Scanner, which is used to create AR experiences.

We mined Apple's acquisitions, investments, and partnerships to discern the company's emerging strategic priorities. Apple is looking for the next big thing. As the world's most valuable company, with a $2T market cap and over $100B in annual profit, Apple has long been known for its cell phones, laptops, tablets, and watches. Today, the company is investing aggressively in high-tech areas like AR/VR, AI, and semiconductors to lay the groundwork for products and features that affect health & wellness, mobility, digital connections, and more. Download our full report to find out the top trends poised to reshape industries in 2022.